impossible video
Intuitive physics understanding emerges from self-supervised pretraining on natural videos
Garrido, Quentin, Ballas, Nicolas, Assran, Mahmoud, Bardes, Adrien, Najman, Laurent, Rabbat, Michael, Dupoux, Emmanuel, LeCun, Yann
We investigate the emergence of intuitive physics understanding in general-purpose deep neural network models trained to predict masked regions in natural videos. Leveraging the violation-of-expectation framework, we find that video prediction models trained to predict outcomes in a learned representation space demonstrate an understanding of various intuitive physics properties, such as object permanence and shape consistency. In contrast, video prediction in pixel space and multimodal large language models, which reason through text, achieve performance closer to chance. Our comparisons of these architectures reveal that jointly learning an abstract representation space while predicting missing parts of sensory input, akin to predictive coding, is sufficient to acquire an understanding of intuitive physics, and that even models trained on one week of unique video achieve above chance performance. This challenges the idea that core knowledge -- a set of innate systems to help understand the world -- needs to be hardwired to develop an understanding of intuitive physics.
- Health & Medicine > Therapeutic Area > Neurology (0.46)
- Education (0.46)
- Information Technology > Artificial Intelligence > Vision (1.00)
- Information Technology > Artificial Intelligence > Natural Language > Large Language Model (1.00)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning (1.00)
- Information Technology > Artificial Intelligence > Cognitive Science (1.00)
Skydio - Impossible Video, Now Possible. Master Data Science
Skydio is an autonomous drone technology company that designs and develops GPS enabled software to navigate drones. Since 2014, Skydio has established its company which specializes in artificial intelligence, robotics and computer vision. Not until 2018 they launched R1 which was considered as a breakthrough in autonomous drones for customers and as a platform for commercial development. Nowadays, they are building autonomous systems, using artificial intelligence to duck under canopies and dive around branches in forests with caution all by itself. Furthermore, not only it is able to avoid obstacles, some other benefits are they provide remarkable pictures no other drones can.